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Section IV: Trends in Teaching Practice 

Excerpted From: Does Professional Development Change Teaching Practice? Results 
From a Three-Year Study 

U.S. Department of Education , Office of the Under Secretary 


In this section, we put our re- 
sults on the effects of profes- 
sional development in context 
by examining how much aver- 
age change over time there is 
in different dimensions of 
teaching practice. In particular, 
we use our three years of data 
on teaching practice to describe 
trends and differences across 
teachers and schools in the 
three broad aspects of teaching 
practice that we discussed in 
Section III: 

• The overall alignment of 
teaching with national stan- 
dards, using the National As- 
sessment of Education 
Progress (NAEP) as the na- 
tional standard 

• Teachers’ emphasis on the 
six performance goals (i.e., 
memorize, understand con- 
cepts, perform procedures, 
generate hypotheses, collect, 
analyze/interpret, and make 
connections) 

• Teachers’ use of the four 
pedagogical strategies (i.e., 
teacher-centered instruc- 
tion, individual seatwork, ac- 
tive instruction, and discus- 
sion-oriented instruction) 

First we report patterns in 
teacher change over the three- 
year period of our study on align- 
ment, performance goal empha- 
sis, and pedagogical strategies. 
Then we analyze differences in 
teaching practice between sub- 
jects and school levels, schools, 
and teachers and for individual 
teachers over time. 

Do Teachers Change 
Their Teaching Practice? 

• In our longitudinal sample, 
we find little change in over- 
all teaching practice from 1996 


to 1999. In particular, there 
was little change in teachers’ 
average degree of alignment 
with the NAEP, in patterns of 
emphasis on performance 
goals, and in pedagogy. 
Exhibit 18 shows the mean, or 
average, levels of emphasis for 
all 1 1 measures of teaching 
practice for each of the three 
years of the study. These mean 
levels of emphasis reflect teach- 
ing practice across all teachers 


Systemic reforms foster mul- 
tiple signals, and many of the 
signals may conflict, which could 
create a tension that causes 
teachers to be resistant to change 
(Cohen & Spillane, 1992). 


in our longitudinal sample. (See 
Appendix E for mean levels of 
emphasis by subject and school 
level.) Using hierarchical linear 
modeling, we tested whether 
there were significant changes 
in these mean levels of empha- 
sis, controlling for subject, 
school level, and the interaction 
of subject and school level. We 
found that generally, teachers’ 
classroom practice remains 
stable across the three years of 
the study. For example, the first 
row in Exhibit 1 8 indicates that 
in 1996-97, on average, 21 per- 
cent of teachers’ content cover- 
age was aligned with the NAEP. 
(This exhibit is identical to Ex- 
hibit 7 in Section III.) Although 
the alignment of content cover- 
age with the NAEP drops to 20 
percent in 1997-98 and 1998- 
99, this difference is not statis- 
tically significant. The exhibit also 
shows that teachers’ average em- 
phasis on higher-order perfor- 
mance goals and topics and on good 
pedagogical strategies also do not 


increase over time. 1,2 

There may be several expla- 
nations for the lack of change 
in overall teaching practice. 
First, it is not unreasonable to 
think that teachers teach in 
ways that they believe to be ef- 
fective and appropriate and do 
not make substantial changes 
in their practice from year to 
year, as a result of either pro- 
fessional development or other 
influences. We would not expect 
other professionals, in fields 
such as law or medicine, to 
make substantial changes in 
their behavior on the basis of 
one or even several professional 
development experiences. Sec- 
ond, teachers may be getting too 
many signals to teach in con- 
flicting ways, and the tensions 
may have the effect of canceling 
each other out. Systemic reforms 
foster multiple signals, and many 
of the signals may conflict, which 
could create a tension that causes 
teachers to be resistant to change 
(Cohen & Spillane, 1992). 3 

It also is possible that 
teachers may be changing, but 
slowly. We measured teaching 
at only three points in time over 
three years, and thus we may 
not have measured teachers 


These results also highlight 
the measurement and theo- 
retical complexities in the 
study of teacher change. 


long enough to capture substan- 
tial change. Further, teachers 
may be changing, but in ways 
not captured by our study. For 
example, teachers may be in- 
creasing their alignment with 
state standards, but not with the 
NAEP. 4 Finally, teachers may be 
shifting in a particular direc- 
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Exhibit 18. 




Average Characteristics of Teachers’ Instruction for 1 996-97, 1 997-98, and 1 998-99. 

Dimension of Teaching Practice 

1996-97 

1997-98 

1998-99 

Alignment of content Coverage with the NAEP 
Scale: From 0 to 100% alignment with the NAEP 

Mean (SD) 

Mean (SD) 

Mean (SD) 

Alignment Index 

21% (9) 

20% (9) 

20% (8) 


Performance Goals 

Scale: From 0 to 100% emphasis on the performance goal 


Memorize 

15% (7) 

16% (6) 

15% 

(6) 

Understand Concepts 

23 % (6) 

22% (5) 

22% 

(5) 

Perform Procedures 

20% (6) 

20% (6) 

20% 

(7) 

Generate Hypotheses 

11% (5) 

12% (5) 

12% 

(5) 

Collect, Analyze, Interpret 

13% (5) 

13% (4) 

13% 

(5) 

Make Connections 

17% (5) 

17% (4) 

17% 

(4) 


Pedagogy 

Scale: Standardized scale where mean=50 and standard deviation=10, 


in 1996-97 

Didactic Instruction 
Individual Seatwork 
Active, Project-centered Instruction 
Discussion-oriented Instruction 


50.1 (5.6) 
49.5(5.4) 
49.7(5.7) 
50.0 (10.8) 


50.6(5.8) 
49.4(5.5) 
49.5(5.8) 
49.6 (9.3) 


49.9 (6.2) 
49.1 (5.6) 
49.1 (6.4) 
49.4(8.7) 


How to read this exhibit: The percent alignment of teachers’ content coverage with the NAEP decreased from 21 
percent in 1996-97 to 20 percent alignment in 1997-98 and 1998-99. These changes are not statistically signifi- 
cant. The percent emphasis that teachers give to the performance goal of memorization increased from 15 percent 
in 1996-97 to 16 percent in 1997-98; it decreased to 15 percent in 1998-99. These changes are not statistically 
significant. Teachers’ use of the pedagogical strategy of didactic instruction increased from 50.1 to 50.6 in 1997- 
98; it decreases to 49.9 in 1998-99. These changes are not statistically significant. Pedagogy is on a standardized 
scale where 50 is the mean level of use of each of the four strategies; this number is not a percent, but rather a 
general metric designed to provide a point from which to show increases or decreases in teachers’ use of a particu- 
lar pedagogical strategy. The mean was standardized in 1996-97, but could vary in 1997-98 and 1998-99. 


tion, but to increase emphasis 
on certain types of performance 
goals or pedagogy, teachers have 
to de-emphasize other strate- 
gies. This may result in the 
overall appearance of no change 
when the data are aggregated, 
when actually changes in one 
direction may offset changes in 
another direction. 5 

These results also highlight 
the measurement and theoreti- 
cal complexities in the study of 
teacher change. There may be 
important differences between 
measures designed to capture 
teachers’ instruction at one 
point in time and measures that 
are designed to detect teacher 


growth (see Rowan, 2000). For 
example, as seen in Section III 
of this report, we were able to 
detect the effects of professional 
development on teachers’ use of 
specific strategies intended to 
foster higher-order student 
learning, but we found relatively 
few effects on global measures 
of teaching. We do not have any 
working paradigms for teacher 
growth — either theoretical or 
empirical frameworks for what 
types of changes one would ex- 
pect, their magnitude, and 
when these changes would oc- 
cur. This is an area where more 
theoretical and empirical work 
is needed. Despite these cave- 


ats, our findings provide strong 
evidence of the stability of 
teachers’ classroom practice. 

Variation in Teaching 
Practice Between Schools, 
Between Teachers Within 
Schools, and Across Time 

Although we observe little 
change in classroom practice on 
average, individual teachers 
differ substantially in their 
classroom practice. We found 
the following: 

• Despite little average change 
over time in teaching prac- 
tice in our longitudinal 
sample, individual teachers 
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Exhibit 19. 

Percent Variation in Teaching Practice (1 996-97, 1 997-98, 1 998-99). 


Dimensions of Teaching Percent of Variation in the Quality of Professional De- Unexplained 


Practice 

velopment Explained by Differences Across: 


Year-to-Year 






Variation in 


Subjects 
and School 
Level 

Teachers 
in the Same 
School 

Schools 

Years 

Individual 

Teaching 

Practice 

Alignment 

Alignment Index 

42.4% 

27.2% 

0.0% 

0.0% 

30.4 

Scale: From 0 to 100% 
alignment with the NAEP 






Performance Goals 






Scale: From 0 to 100% emphasis 
Memorization 

on the performance goal 

0.0% 39.1% 

2.2% 

0.0% 

58.7 

Understanding concepts 

0.0 

30.0 

2.5 

0.0 

67.5 

Performing procedures 

27.9 

29.7 

0.0 

0.0 

44.2 

Generating hypotheses 
Collecting/ analyzing/ 

6.7 

40.0 

0.0 

0.0 

53.3 

interpreting data 

10.3 

34.5 

0.0 

0.0 

55.2 

Making connections 

3.7 

29.6 

0.0 

0.0 

66.7 

Pedagogy 

Scale: Standardized scale where 
Traditional 
Individual seatwork 
Active project-centered 

mean=50 and standard 

25.3 

10.1 

deviation=10, 

40.9 

40.6 

in 1996-97. 
6.2 
9.6 

0.0 

0.4 

27.6 

39.3 

learning 

Discussion-oriented 

16.7 

36.2 

8.6 

0.0 

38.5 

instruction 

2.6 

30.7 

4.8 

0.0 

61.8 


How to read this exhibit: Forty-two (42) percent of the variation in alignment is due to teachers’ subject and school 
levels; none of the variance is due to differences between schools or average differences in teaching practice across 
years; 27 percent of the variation is between teachers within the same school, and 30 percent of the variation is 
unexplained year-to-year variation. Thus, teacher and subject/level differences account for most of the variance, 
and the remaining 30 percent is unexplained year-to-year variance. 


in our sample do vary in their 
classroom practices, and 
moderate variation does oc- 
cur in the classroom practice 
of individual teachers from 
year to year. 

Although there is very little 
change across years in average 
teaching practice, there still are 
considerable differences be- 
tween teachers, as illustrated by 
the standard deviations reported 
in Exhibit 18. For example, the 
exhibit indicates that on aver- 
age, teachers gave 15 percent 
of their instructional emphasis 
to the performance goal of 


memorization in 1996-97. The 
standard deviation of 7 percent 
indicates that many teachers 
gave as little as 8 percent em- 
phasis to this performance goal, 
whereas many others gave 22 
percent emphasis to this goal. 
Similarly, if we look again at 
alignment, the exhibit shows 
that in 1996-97, the average 
alignment of teachers’ content 
coverage with the NAEP was 2 1 
percent. The standard deviation 
of 9 percent, however, indicates 
that for many teachers, only 12 
percent of content coverage was 
aligned with the NAEP, whereas 


for many other teachers, 30 per- 
cent of their content coverage 
was aligned with the NAEP. So, 
although we observe consider- 
able stability in classroom prac- 
tice on average, individual 
teachers differ substantially in 
their classroom practice. 

Our longitudinal data enable 
us to answer many questions 
about the variation in teachers’ 
classroom practice. For ex- 
ample, do teachers who teach 
different subjects or levels of 
school have different patterns of 
content alignment, perfor- 
mance goals, and pedagogy? Do 


28 

teachers in some schools differ 
systematically in their class- 
room practice from teachers in 
other schools? Do teachers 
within the same school differ 
from one another in their class- 
room practice? 

Our longitudinal data also 
enable us to answer questions 
about changes in individual 
teachers’ instruction over time. 
Are teacher differences consis- 
tent across years? For example, 
do individual teachers who em- 
phasize active, project-centered 
instruction in 1997-98 also 
emphasize this aspect of their 
classroom practice in 1998-99? 

Using a statistical tech- 
nique called “hierarchical lin- 
ear modeling” we were able to 
use our longitudinal data to es- 
timate what proportion of the 
“variation” in teachers’ class- 
room practice (i.e., the 11 as- 
pects of teaching that we mea- 
sure) could be attributed to (1) 
average differences between 
teachers who teach different 
subjects or levels of school, (2) 
average differences between 
schools, (3) average differences 
between teachers in the same 
school, and (4) average differ- 
ences between one year and the 
next. (See Appendix E for an ex- 
planation of the analyses un- 
dertaken for this section of this 
report.) For each dimension of 
teaching practice shown in the 
first column of Exhibit 19, we 
show the percent of the varia- 
tion in teachers’ professional 
development experiences that 
we can attribute to each of 
these differences. 

The last column of Exhibit 
19, labeled “unexplained year- 
to-year variation in individual 
teaching practice,” shows the 
percent of the variation in indi- 
vidual teachers’ practice be- 
tween years that cannot be ex- 
plained by any of the differences 
listed above. If teachers’ class- 
room practice were totally con- 
sistent from one year to the 


next, all of the variation in their 
instruction would result from 
differences between teachers’ 
subject and school levels; differ- 
ences between teachers in the 
same school; differences be- 
tween schools; and differences 
in the average teaching prac- 
tice between one year and the 
next. None of the variation 
would be left unexplained. 

The data reported in Exhibit 
19 indicate that depending on 
the specific dimension of teach- 
ing practice, a substantial part 


Despite the consistency of 
teaching practice over time, 
we found variation between 
individual teachers. 


of the variation in teaching 
practice can be explained by dif- 
ferences in teachers’ subjects 
and school levels. For example, 
over 40 percent of the variation 
in alignment is due to subjects 
and levels. This may reflect the 
fact that, for example, elemen- 
tary school mathematics in- 
struction is more aligned with 
the NAEP than are other sub- 
jects and grades. (See Appendix 
E.) The exhibit also shows that 
almost 30 percent of the varia- 
tion in emphasis on the perfor- 
mance goal “performing proce- 
dures” is due to subjects and 
levels. This supports our earlier 
finding from the analysis of our 
first year of longitudinal data, 
reported in Garet et al., 1999, 
that teachers of mathematics 
emphasize this performance 
goal significantly more than 
other teachers. Finally, about 25 
percent of the variation in 
teachers’ emphasis on tradi- 
tional pedagogy is explained by 
teachers’ subjects and school 
levels. This probably reflects the 
fact that high schools tend to 
have more traditional pedagogy 
than do middle or elementary 
schools. Apart from these differ- 
ences between teachers’ sub- 
jects and school levels, most of 
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the variation in teaching prac- 
tices is between teachers 
within the same school. (See 
Appendix E for a discussion of 
these results.) 

There is practically no varia- 
tion across schools in align- 
ment and performance goal 
emphasis, and there is a small 
difference across schools in 
pedagogy. This difference may 
be due to the adoption of whole- 
school reform designs, which 
often focus on pedagogical strat- 
egies (e.g., some designs hinge 
on project-centered learning). 

Finally, Exhibit 19 indicates 
that there is essentially no year- 
to-year variation in average 
teaching instruction. This re- 
flects our earlier finding of little 
change in average overall 
teaching practice over time. Al- 
though there is substantial 
variation across teachers in 
their teaching practice, this 
variation can be attributed to 
differences across individual 
teachers in the same school to 
teachers who teach mathemat- 
ics instead of science, or to 
teachers who teach at different 
levels of school. 

A substantial amount of 
year-to-year variation in the 
teaching practice of individual 
teachers remains unexplained 
in our analysis. This unex- 
plained year-to-year variance is 
higher for the six performance 
goals than for alignment or for 
the four measures of pedagogy. 
The high year-to-year variation 
in emphasis on performance 
goals may indicate that teach- 
ers adjust these goals to 
changes in the specific student 
composition of their classes 
each year. 6 

Summary 

Although teachers may be 
changing on dimensions or 
qualities of practice that we did 
not measure, it is evident that 
on many central dimensions of 
classroom practice, teachers in 
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the 30 schools we studied did not 
change from 1996-97 through 
1998-99. However, despite the 
consistency of teaching practice 
over time, we found variation 
between individual teachers. 
We found differences in teach- 
ing practice by school level and 
subject, and we found that most 
of the variation in teaching 
practice is between individual 
teachers within the same 
schools, rather than between 
schools. Greater differences in 
teaching practice between 
schools might indicate consis- 
tent, systematic school-level 
instructional plans, but evi- 
dence of such planning was not 
found in our data. 

Our findings highlight the 
value of conducting studies with 
increased power to measure 
change (for example, measuring 
teachers for more than three 
years). The findings also high- 
light the value of studies that 
incorporate models of teacher 
growth that would indicate what 
types of changes we would ex- 
pect in teaching practice, as 
well as their timing and mag- 
nitude. For example, it would be 
helpful to have information on 
the type of dimensions on which 
we expect teachers to change 
and by how much we would ex- 
pect them to change during a 
one-year period versus a two- 
year period. 

Lastly, although teachers do 
not report changing their teach- 
ing practice in ways we might 
consider desirable, the fact that 
they are consistent over a 
three-year period in reporting 
their instructional practices lends 
strong support to our survey instru- 
ment as a reliable measure of 
teachers’ instruction. 

In the final section of this 
report, we summarize and syn- 
thesize our findings on teach- 
ers’ professional development 
experiences and the effects of 
professional development on 
instruction and trends in teach- 


ing practice, and we suggest 
implications for designing and 
supporting professional develop- 
ment through the Eisenhower 
and other programs. 

References 

Beaton, A. E., Mullis, I. V. S., Mar- 
tin, M. O., Gonzalez, E. J., Kelly, 
D. L., & Smith, T. A., (1996). 
Mathematics acheivement in the 
middle school years: IEA’s Third 
International Mathematics and Sci- 
ence Study (TIMSS). Chestnut 
Hill, MA: TIMSS International 
Study Center. 

Cohen, D. K., Spillane, J. (Eds.) 
(1992). Policay and practice: The 
relations between governance and 
instruction. Washington, DC: 
American Educational Research 
Association. 

Garet, M., Birman, B., Porter, A., 
Desimone, L., & Herman, R. with 
Suk Yoon, K. (1999). Designing ef- 
fective professional development: 
Lessons from the Eisenhower Pro- 
gram. Washington, DC: U.S. De- 
partment of Education. 

The previous excerpt is Section 
IV of the following document: 
U.S. Department of Eduction, 
Office of the Under Secretary, 
Planning and Evaluation Ser- 
vice, Elementary and Secondary 
Education Division, Does Profes- 
sional Development Change 
Teaching Practice? Results from a 
Three-Year Study, Executive Sum- 
mary, Washington, D.C, 20202. 

Endnotes 

1 At the outset of the study we 
targeted several topics to 
monitor over time. These 
were subjects, such as prob- 
ability, statistics, measure- 
ment, and geometry, that 
have been identified as spe- 
cial weaknesses for students 
in the United States (Beaton 
et al., 1996). We conducted 
analyses of these and an ex- 
haustive list of topics by sub- 
ject and school level, and the 
only topic for which there was 
even a marginally significant 
increase was advanced alge- 
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bra in high school (p<.07, 
n=28). These analyses were 
done separately by school 
level and subject and had 
sample sizes of approximately 
30 teachers, which limits the 
power to detect effects. 

2 The only significant change 
over time effect is an inter- 
action between year and sub- 
ject for teachers’ emphasis on 
memorizing and understanding 
concepts. See Appendix E for the 
results of these analyses. 

3 We observed during our site 
visits in 1996-97 that many 
of the schools were engaged 
in multiple reform efforts. 

4 AIR is currently conducting a 
study, Moving Standards to the 
Classroom, sponsored by the 
Planning and Evaluation Ser- 
vice, which is focused on ex- 
amining the extent to which 
teachers’ instruction in 
mathematics is aligned with 
state standards; in addition, 
the Wisconsin Center for Edu- 
cation Research, in collabo- 
ration with the Council of 
Chief State School Offices 
(CCSSO), has conducted a 
study of the alignment of 
teaching with state assess- 
ments and is planning a study 
of the alignment of teaching 
with state standards. 

5 This also might help explain 
the random variation in in- 
dividual teachers’ trajecto- 
ries that we found. 

6 The fact that only 30 percent 
of the variation in alignment 
is due to unexplained year-to- 
year variation indicates that 
the “test-retest” reliability of 
our measure of content cov- 
erage is relatively high. If all 
the year-to-year variation 
were due to measurement er- 
ror, the implied reliability 
would be .70. 


